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Abstract 

T his study analyses some published interactive materials for the 
learning of Spanish as a first language and English as a Foreign 
Language (EFL) commonly used in primary and secondary education 
in Spain. The present investigation looks into the relationships between 
text and image on the interface of Interactive Digital Material (IDM) to 
develop learners’ language skills. Screen design is evaluated with regards 
to the following formal units of analysis: graphic elements (shape, colour, 
size, resolution, significance), typography (style, colour, size, readability), 
composition (location, ratio) and action (recognition and effects) to assess 
their functionality in various learning activities. A discussion is also 
presented on the way these features of multimodal discourse can influence 
the language learning processes. 
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1. Introduction 

The growing publication of digital educational materials for language learning 
in primary and secondary school is mainly due to two factors. On the one hand, 
it is logical that the teaching-learning situations benefit from all the advantages 
that digital environments provide in their multiple facets. On the other hand, it is 
evident that twenty-first century children and youngsters are active participants 
in the so called digital era, which entails that the forms of communication which 
they are now engaged in have changed considerably to previous generations, and 
therefore, an adjustment is needed to formalise the contents of digital materials 
to the new needs of the users. 

Taking this into consideration, a key dimension that differentiates digital materials 
is the concept of interactivity. In this context, interactivity is identified as the 
relationship established by the users with the interface to process information 
actively, and thus increase learners’ motivation, which results in more effective 
learning. 

From this perspective and in the framework of a wider research tackling 
interactive material for the learning of Spanish as a first language and EFL, the 
present study looks in particular into the relationships between text and image 
on the interface of IDMs for language learning. 

The analysis of screen design deals with the following formal units and features 
(in parenthesis): graphic elements (shape, colour, size, resolution, significance), 
typography (style, colour, size, readability), composition (location, ratio), and 
action (recognition and effects). Besides, their functionality in language learning 
activities is also discussed. 

The study of the design of interactive digital materials for language learning 
is a research issue that should be addressed from different perspectives. To 
ensure meaningful interactivity between the learner and the digital material, 
courseware design should be analysed taking into account the usability of the 
learning material from the study of multimodal discourse (Han, Yun, Kwahk, 
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& Hong, 2001; Marzal, Colmenero, & Morato, 2003; Nielsen & Morkes, 1998; 
Nokelainen, 2006). 


2. The concept of interactivity 

The concept of interactivity is investigated from several different fields, such as 
Human Computer Interaction (HCI), instructional design, artificial intelligence, 
e-learning, or multimedia. From the instructional design research, some studies 
indicate interactivity has a positive influence on learning (Najjar, 1998; Ohl, 
2001; Robertson, 1998; Sims, 1997; Yacci, 2000) and motivation (Stocks 
& Freddolino, 2000; Teo, Oh, Liu, & Wei, 2003). Increased user control, as 
reported by Brady (2004), “should increase learning and [...] satisfaction. 
Similarly, increasing active processing should result in increased learning 
outcomes” (para. 2). Thus, it is crucial to study the interactivity of IDMs to 
understand how it applies, how it can enhance the process of learning and when 
better performance can be achieved to improve this process. 

Will students learn more and better with IDMs? With the same content, IDMs 
have more potential to promote learning and language acquisition than traditional 
materials may provide. The answer lies in interactivity, which can capture the 
attention of learners and motivate them to learn. Students are asked to act at 
various times, keeping their attention throughout the whole process. 

Sims (1997) classifies interactivity in various non-exclusive concepts (object, 
linear support, construct, reflexive, simulation, hypertext, contextual without 
immersion and virtual immersion), which refer to various instructional 
tasks and help understand the relationship between the digital material and 
the learner. Later, Sims (2000) analyses the elements of online interactivity 
through the four key components of any instructional design: the learner, 
the content, the pedagogy and the context. Furthermore, Kennewell, Tanner, 
Jones, and Beauchamp (2008) distinguish between two types of interactivity: 
technical and pedagogical. The first one refers to the relationship between a 
device (such as an Interactive Whiteboard (1WB), a tablet, or a computer) 
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and the student, and the second one refers to the relationship of teacher- 
students through a strategy of content teaching in which the latter are active 
participants. 

In the teaching and learning of languages, the four relations established by Moore 
(1993) are key to the development of communicative competence of students. 
The interactivity of the student and the material, the student and the interface, 
and the interaction between students and teacher and among students themselves 
generate enough practice to develop learners’ communicative skills. 

In a digital environment, teachers need to develop a particular pedagogy about the 
use of the material either for the IWBs, computers, tablets or iPads', that is, when 
they can be used and what for. This decision is determined by the characteristics 
of the digital materials available for teachers. lDMs are, therefore, a powerful 
tool for teaching and learning that acts as medium, which, in turn, allows for 
integration of other tools, such as the ones provided by the so-called web 2.0 
(Sessoms, 2008). 

The new spaces of communication and interaction in web 2.0 imply substantial 
changes in the use of language (Yus, 2010). The language of the Internet is 
characterised by its multimodality, i.e. by the use of different modes for effective 
communication. In this sense, that must also be the subject of teaching and 
practice, according to the new interpretation and production processes involved 
in digital literacy (Cassany, 2011). 


3. Multimodal discourse of the interface 

Language is defined as a code, a system structured in ways that mean something 
in an independent medium, either from written text, music or art (Barthes, 1970). 
However, a new approach considers now that the codes are structured resources 
in the same message. The message is the meeting point of different codes. Thus, 
a mode is a means of expression used to convey meaning. Each message uses a 
number of modes thus becoming multimodal. 
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Following the social semiotics theory (Bum & Parker, 2001), multimodal 
discourse text analysis (where text here must be understood as any written or 
graphic document) is expressed according to a series of parameters and must 
consider three social functions: a) the representation of an aspect of reality, 
b) the orientation in which relationships are established between interlocutors 
(either real people, fictional or between fictional agents and readers), and c) the 
way in which communication is organised with consistency and structure from a 
conceptual unit and a structural one. 

Kress and van Leeuwen (2001) pioneered a proposal of a grammar of the visual 
based on Halliday’s (1994) model. According to Halliday (1994), every text 
is a multifunctional semantic unit that produces meanings at three levels: one 
related to the ideas expressed by the ideational function, the other that refers 
to the attitudes of the addresser about the message and the addressee using 
the interpersonal function, and the last one refers to the linguistic structure of 
text expressed in the textual function. Thus, Kress and van Leeuwen (2001) 
translated this three-level model: ideational, interpersonal and textual to a three- 
dimensional visual grammar. The visual mode extends in three comparable 
mode dimensions: (a) the ideational function, as the ability of visual resources 
to represent objects and situations of the world; (b) the interpersonal function, 
understood as the ability of visual language to express the relationship between 
the producer and the recipient of a sign; and (c) the textual function, in which 
visual resources combine to form a grammar. 

From the above it follows that the degree of interactivity of digital materials is 
determined largely by the quality of the interface design. Design is understood 
here as the adequacy of the formal presentation of the content (text and image) to 
develop optimal performance in the processes, phases and sequences of various 
activities. Therefore, the interface is a space in which the effectiveness of 
multimodal discourse will be successful if the fonnal components that structure 
the display follow some quality criteria that are determined by their functionality. 

As far as digital materials for language learning are concerned, interactivity 
enhanced learning is determined by the ease and functionality of the management 
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processes in the development of activities, which in turn are determined by the 
quality of the design elements that structure the screens. 


4. Analysis of screen design: measuring interactivity 
from a multimodal discourse perspective 

The quality of graphic design is a key element to facilitate interactive processes 
in the development of digital activity. The formalisation of typography and 
image, and the relationship between the two (i.e. multimodality) may influence 
the reception of messages and facilitate or hinder interactivity. 

To assess the interactivity that screens can generate in a didactic sequence of 
digital material, four aspects of design are considered. 

The first one is a graphic aspect, that is to say, the iconographic elements that 
compose the screen image. The categories analysed are 

• Shape: all the features that define the appearance of iconographic 
elements (technical realisation, style, perspective, proportion, etc.); 

• Colour, chromatic properties of the iconographic element (hue, 
saturation, lightness); 

• Size', proportions of the iconographic elements in relation to the space 
they occupy; 

• Resolution', quality in the definition of the screen image; 

• Significance', relevance and adequacy of iconographic elements in 
relation to the content. 

The second aspect is typographic, and is analysed by means of the following 
components: (a) style (properties that define the look of the typographic element, 
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that is, family, font, etc.); (b) colour, (c) size', and, last but not least, (d) readability 
(clarity in text reading). 

The third aspect is composition, i.e. the relationship between the different 
elements, whether iconographic, typographical, or both simultaneously, which 
make up the screen image. Two categories are studied here: 

• Location', place where the different elements are located; 

• Proportion', harmony between elements regarding their dimensions. 

The fourth aspect is devoted to the elements of action, i.e. graphic or typographic 
elements that must be activated to interact with the screen and allow the user to 
select, start, fast-forward, rewind, close, etc. The categories considered in this 
aspect are 

• Recognition', identifying that the graphic or typographical elements 
will cause an action; 

• Visual effects (if any): when activating the element, variations in colour, 
size, and lighting are displayed; 

• Sound effects (if any): when activating the element, sounds are emitted. 

The main criteria by which the different categories are analysed are functionality 
and consistency. Therefore, those components that hinder the interactive 
processes are identified. Interactivity impedes the teaching-learning processes 
in the graphic dimension when, for instance, the images have only a ‘decorative’ 
function, the colour combination is strident, or poor resolution prevents the 
proper display of the represented objects. Similarly, in the typographic aspect, 
the font chosen or an excessively small size may hinder readability, as may 
a colour that does not stand out enough from the background. Likewise, at a 
compositional level, images and texts can be pressed to each other or not 
distributed harmoniously. And finally, regarding the elements of action, they 
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might be difficult to identify, and visual or sound effects (if any) might be 
gratuitous or too distractive. 

Each screen is considered a ‘unit of analysis’, and the categories defined above 
are described for each one. However, independent analysis of each screen is 
not sufficient to assess the interactive processes, as activities are formed into 
a chained sum of screens. Thus, another criterion is introduced to analyse the 
coherence between screens of the same material. On the one hand, the existence 
of coherence between screens of the same unit (the subsequent unit of the 
analysed material) and, on the other hand, between the screen that is evaluated 
and considered to serve a similar role in the second unit analysed. 

Examples of interactivity that impede teaching-learning processes are baffling 
the user with unexpected typographic, chromatic and compositional changes, or 
random changes occurring in the shape and location of an element of action in 
the same unit screens or with similar function. 


5. Conclusion 

In the interactive processes come into play several factors, such as learner, 
content, pedagogy and context (Sims, 2000). In the present study of language 
learning in digital environments, the learner should receive content from a 
clear and precise multimodal discourse so that interactivity can prompt 
learning. Clarity of content is key both in the educational design of the activity 
and its formal aspect of the design. In this sense, graphic design affects not 
only aesthetic considerations, but focuses primarily on issues related to 
functionality. 

The main function of IDMs for language learning is, obviously, to learn and 
practise language using an active methodology in which interactivity promotes 
learning from effective multimodal discourse. The multimodal discourse is 
generated on the screen from the established interrelation between the written 
text and image. In this sense, graphic design directly affects the interactive 
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processes, not only regarding the instructions of the activities but also the 
learning of the content. 

In summary, analysing the criteria related to (a) the formal presentation of screens 
(graphic elements, typography, composition, and action); and (b) the coherence 
in IDMs, determines the quality of their interactivity and, consequently, their 
effectiveness for language learning. 
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